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The mathematical theory of compressed sensing (CS) asserts that 
one can acquire signals from measurements whose rate is much 
lower than the total bandwidth. Whereas the CS theory is now well 
developed, challenges concerning hardware implementations of 
CS-based acquisition devices — especially in optics — have only 
started being addressed. This paper presents an implementation 
of compressive sensing in fluorescence microscopy and its ap- 
plications to biomedical imaging. Our CS microscope combines 
a dynamic structured wide-field illumination and a fast and sensi- 
tive single-point fluorescence detection to enable reconstructions 
of images of fluorescent beads, cells and tissues with undersam- 
pling ratios (between the number of pixels and number of mea- 
surements) up to 32. We further demonstrate a hyperspectral 
mode and record images with 128 spectral channels and under- 
sampling ratios up to 64, illustrating the potential benefits of CS 
acquisition for higher dimensional signals which typically exhibits 
extreme redundancy. Altogether, our results emphasize the inter- 
est of CS schemes for acquisition at a significantly reduced rate 
and point out to some remaining challenges for CS fluorescence 
microscopy. 

compressive sensing | sparse images | fluorescence microscopy | biolog- 
ical imaging 

Abbreviations: CS, compressed sensing; CFM, compressive fluorescence mi- 
croscopy 

Introduction 

Fluorescence microscopy is a fundamental tool in basic and ap- 
plied biomedical research. Because of its optical sensitivity 
and molecular specificity, it is employed in an increasing number 
of applications which, in turn, are continuously driving the devel- 
opment of advanced microscopy systems that provide imaging data 
with ever higher spatio-temporal resolution and multiplexing capabil- 
ities. In fluorescence microscopy, one can schematically distinguish 
two kinds of imaging approaches, differing by their excitation and 
detection modalities [1]. In wide-field (WF) microscopy, a large sam- 
ple area is illuminated and the emitted light is recorded on a multi- 
detector array, such as a CCD camera. In contrast, in raster scan (RS) 
microscopy, a point excitation is scanned through the sample and a 
point detector is used to detect the fluorescence signal at each posi- 
tion. 

While very distinct in their implementation and applications, 
these imaging modalities have in common that the acquisition is inde- 
pendent of the information content of the image. Rather, the number 
of measurements, either serial in RS or parallel in WF, is imposed by 
the Nyquist-Shannon theorem. This theorem states that the sampling 
frequency (namely the inverse of the image pixel size) must be twice 
the bandwidth of the signal, which is determined by the diffraction 
limit of the microscope lens equal to A/2NA (A is the optical wave- 
length and NA the objective numerical aperture). Yet, most images, 
including those of biological interest, can be described by a num- 
ber of parameters much lower than the total number of pixels. In 
everyday's world, a striking consequence of this compressibility is 



the ability of consumer cameras with several megapixel detectors to 
routinely reduce the number of bits in a raw data file by an order 
of magnitude or two without substantial information loss. To quote 
from David Brady: "if it is possible to compress measured data, one 
might argue that too many measurements were taken" [2]. 

The recent mathematical theory of compressed or compressive 
sensing (CS - see [3, 4]) has addressed this challenge and shown how 
the sensing modality could be modified to reduce the sampling rate 
of objects which are sparse in the sense that their information content 
is lower than the total bandwidth or the number of pixels suggest. 
The fact that one can sample such signals non-adaptively and without 
much information loss — if any at all — at a rate close to the image in- 
formation content (instead of the total bandwidth) has important con- 
sequences, especially in applications where sensing modalities are 
slow or costly. To be sure, the applications of CS theory to data ac- 
quisition are rapidly growing in fields as diverse as medical resonance 
imaging [5, 6], analog-to-digital conversion [7, 8] or astronomy [9]. 

In optics, the interest in CS has been originally spurred by the 
demonstration of the so-called "single-pixel camera" [10], Since 
then, reports have explored the potential of CS for visible and in- 
frared imaging [11, 12], holography [13] or ghost imaging [14]. In 
microscopy, the feasibility of CS measurements has recently been 
demonstrated [15]. Altogether, these results open exciting prospects, 
notably for the important case of biomedical imaging. Having said 
this, there are very few results about the performance of CS hardware 
devices on relevant biological samples. As such samples often have 
low fluorescence, it is especially important to understand how the as- 
sociated noise will affect the acquisition and reconstruction schemes. 

In this paper, we describe Compressive Fluorescence Microscopy 
(CFM), a novel modality for fluorescence biological and hyperspec- 
tral imaging based on the concepts of CS theory. In CFM, the sample 
is excited with a patterned illumination and its fluorescence is col- 
lected on a point detector. Images are computationally reconstructed 
from measurements corresponding to a set of appropriately chosen 
patterns. Therefore, CFM benefits from many advantages associated 
with RS techniques, namely, high dynamic range, facilitated multi- 
plexing, and wide spectral range (from the UV to the IR). In truth, 
the benefits of CS are particularly appealing in biology where fast, 
high-resolution and multicolor imaging is highly sought after. 

The paper is organized as follows. We begin by recalling the 
principles of CS theory for optical imaging. We then turn to the de- 
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scription of the practical implementation of CFM and of the sens- 
ing protocol. Our techniques are subsequently applied to image sev- 
eral relevant samples, including fluorescent beads, cultured cells and 
tissues. By extending our implementation, we further demonstrate 
the possibility of hyperspectral acquisition with up to 128 different 
spectral channels. A final contribution is a careful study of various 
noise trade-offs for CFM. We conclude the paper with a discussion 
of prospective CFM developments. 



Compressed Sensing Framework 

We wish to image a two-dimensional sample x = {a:[i]}, a distri- 
bution of fluorescent probes, in which x[i] is the value of x at the 
pixel/location i (thus one can view pixel intensities x[i] as the coef- 
ficients of the image x in a basis of localized functions, namely, the 
Dirac basis). We represent this object in a basis W of our choosing 
and write 

x = c N w p = Wc > 

p 

where the w p 's are (orthogonal) basis functions and the c[p]'s are the 
coefficients of x in the expansion. We say that the signal is A-sparse 
if at most A of these coefficients are nonzero. An empirical fact is 
that most images of interest are well approximated by A'-sparse ex- 
pansions with A much less than the number of pixels N, and this is 
the reason why data compression is effective; one can store and trans- 
mit quantizations of the large coefficients, ignore the small ones, and 
suffer little distortion. 

In our imaging setup, we measure correlations between the im- 
age of interest x and sensing waveforms 0k taken from another basis 
4>; that is, we measure 



y k = (x,0 k ) =^2x[i]4> k [i]. 



[1] 



Here, 0k is an illumination or intensity pattern so that y k is obtained 
by collecting all the fluorescence corresponding to those pixels that 
have been illuminated on a single-point detector. Wide-field and 
point-like excitation are two extreme cases, corresponding respec- 
tively to a uniform sensing waveform (0/j [i] = 1 for all i) and to a 
spike or Dirac waveform. 

In its simplest form, CS theory asserts that if the signal x is sparse 
in the representation W, then only few measurements of the form [ 1 ] 
are sufficient for perfect recovery provided the sensing and represen- 
tation waveforms, respectively 0k and w p , are incoherent [4, 16]. 
Two systems are said to be incoherent if any element in one of the two 
cannot be expressed as a sparse linear combination of elements taken 
from the other. Formally, the coherence between two orthobases W 
and <l? of M. N is measured by the parameter ji/(W, 3>) ranging be- 
tween 1 and N: 



KW,*) = JVxnax|<Wp,0 k )r 

p,k 



[2] 



The Fourier and Dirac bases are in this sense maximally inco- 
herent (we need many spikes to synthesize a sinusoid and vice versa) 
and fj, = 1, On the opposite, two identical bases are maximally 
coherent and, in this case, y, = N. Hence, incoherence expresses 
the idea of the level of dissimilarity between any two representations 
of a signal. With this in mind, one perceives how each incoherent 
measurement — a projection on an element of the basis 4> — conveys 
a little bit of information about all the entries of the coefficient vector 
c. An important result in CS theory states that A'-sparse signals can 
be recovered exactly from comparably few measurements in an in- 
coherent system. Further, recovery is achieved by solving a tractable 
optimization program — a linear program. One solves 



mm 



|c||< x subject to y k = (0 k , Wc), for all k = l...M. [3] 



When M measurements are chosen uniformly at random from the 
basis <t>, the recovery is exact with very high probability; that is, the 
solution sequence c obeys x = ^ p c[p]w p = x, provided that 



M > C W) A log N, 



[4] 



where C is a constant on the order of unity. This result emphasizes 
both the role of the coherence and the potential gain for large images 
due to the logarithmic dependence in the pixel size. For incoherent 
pairs, we only need on the order of A log N random samples. 

We have discussed sparse signals above for ease of exposition. 
However, the theory extends to approximately sparse signals and to 
noisy data. For instance, if the signal is well approximated by a A- 
sparse signal (some would say that it is compressible), then the re- 
construction error is shown to be small. Further, the recovery is not 
sensitive to noise in the sense that the error degrades gracefully as 
the signal-to-noise ratio decreases. We refer to [17] and references 
therein for quantitative statements. 

Compressive Fluorescence Microscopy: Implementation 

Experimental setup. Our setup is based on a standard epifluores- 
cence inverted microscope (Nikon Ti-E) as shown in Figure A. To 
generate spatially modulated excitation patterns, we incorporated a 
Digital Micromirror Device (DMD) in a conjugate image plane of 
the excitation path. The DMD is a 1024-by-768 array of micromir- 
rors (Texas-Instrument Discovery 4100, Vialux, Germany) of size 
13.68x13.68 /im each, and which can be shifted between two posi- 
tions oriented at +12° or -12° with respect to the DMD surface. They 
are all independently configurable at frequency up to 20 kHz. The 
DMD is carefully positioned so that the optical axis (defined by the 
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Fig. 1. (A) Experimental set-up. The (lotted and plain segments corre- 
spond to planes respectively conjugated to the pupil and sample planes. 
(B) Slice of lily anther (endogenous fluorescence with epifluorescence 
microscopy image recorded on a CCD camera). (C) Projection of a 
Hadamard pattern on a uniform fluorescent sample. (D) Projection of 
the same Hadamard pattern on the biological sample. (£) Fluorescence 
intensity during an acquisition sequence. 
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microscope lens and the dichroic mirror DM (Figure A)) is orthogo- 
nal to the plane of the DMD. 

As light source, we used a laser bench (Roper Scientific, France) 
equipped with two superimposed continuous-wave laser (A=488 nm, 
Coherent, USA and A=561 nm, Cobolt, Sweden). The laser beam 
first passed through a rotating diffuser or a phase scrambler (Dyop- 
tika, Ireland) in order to reduce the spatial coherence and was then 
coupled to a 200/im multimode fiber. At the fiber output, the laser 
beam was expanded into a 2 cm diameter collimated beam. This 
beam was oriented towards the DMD at an angle of incidence corre- 
sponding to twice the tilting angle of the DMD mirrors (approx. 24°); 
a micromirror oriented at +12° would reflect the light into the micro- 
scope and appear as a bright pixel in the sample plane and, inversely, 
micromirrors oriented at -12° appear as dark pixels. Depending on 
the samples, we used an air (Nikon, 20X, Plan Apo VC NA 0.75) 
or an oil-immersion objective (Nikon, 60X Plan Apo Tirf, NA 1.45). 
When the 20x air lens was used, the imaging lenses (lenses fi, f-2, fs 
in Figure A) were chosen to introduce a 1.5X reduction. The overall 
magnification of the image of the DMD on the sample was 1/30 and 
the size of a single micromirror equal to 456 nm. When the 60x lens 
was used, a different set of imaging lenses was chosen which only 
served as a lx relay; here, the image size of a single micromirror in 
the sample plane was equal to 228 nm. Upon illumination with an 
intensity pattern (excitation intensity ~ 20 to 60 W/cm 2 ), the sample 
fluorescence was detected on a photomultiplier tube PMT (Hama- 
matsu) and sampled at 96kHz using an analog-digital converter board 
(PCI-4462, National Instruments, USA) (Figure B-D). In CS mea- 
surements, the information on the sample is thus contained in the 
variations of the intensity signal as a function of the illumination pat- 
tern (Figure E). The WF image of the sample could also be directly 
formed on a camera (ImagEM, Hamamatsu, Japan) placed on the out- 
put port of the microscope. For hyperspectral imaging, the PMT was 
replaced by a fast and sensitive spectral detector described later in the 
paper. Note that the role of the DMD in our set-up differs from that 
in the "single-pixel camera" [10] or in some other microscopy setups 
[15]. In the latter, the modulator is placed between the sample and the 
detector, meaning that it is used to select some of the light within the 
total signal, rather than to control the excitation pattern. Our choice 
is motivated by the low level of fluorescence encountered in biolog- 
ical samples such as living cells labelled with fluorescent proteins. 
Indeed, the overall efficiency of a DMD is 68% and falls down to 
34% when only one half of the mirrors are tilted. In our case, the 
photon collection efficiency is only limited by the numerical aperture 
of the microscope lens and the quantum yield of the detector as in 
conventional epifluorescence microscopy. 



Choice of the illumination patterns. For the practical implemen- 
tation of a CS-based image acquisition system, it is essential to de- 
termine which incoherent basis should be used when no prior infor- 
mation on the signal is available. There are measurement ensembles, 
such as the partial Fourier or Hadamard systems, known to be highly 
incoherent with the bases in which most natural images are sparse. 
When excitation patterns are generated by micromirrors, <f>k [i] is a 
binary waveform taking on the two values or 1. An appealing 
choice for <& is then the Hadamard system known to be incoherent 
with the Dirac basis and fine scale wavelets. Since each entry of a 
Hadamard pattern hk is either —1 or +1, one defines </>k as a shifted 
and rescaled version of hk via (pk = (hk + l)/2, which can be sim- 
ply encoded on the DMD. We used patterns of size 256 x 256 and 
128 x 128 obtained by binning 2x2 and 4x4 groups of micromir- 
rors. The actual pattern h k formed in the sample plane is in fact the 
convolution of the ideal pattern hk with the point spread function of 
the microscope P exc in the excitation path. Figures C and D represent 
WF images of a Hadamard pattern projected on a uniform and on a 
biological sample. A specificity of optical imaging is that the sens- 

Footline Author 



ing elements 0k represent light intensities and are thus nonnegative 
which, as discussed later, has important practical implications. 

Hadamard waveforms have a sort of spatial frequency (like si- 
nusoids) which grossly depends on the typical block size of the pat- 
terns. As the power spectrum of most biological images is generally 
concentrated at low frequencies, the flexibility in frequency selection 
is important. We introduce two distinct pattern selection strategies 
based on the expected spatial content of the sample: 

• When the sample we wish to acquire is sparse in the pixel do- 
main as in the case of single molecule or bead imaging, no typical 
frequency range needs to be favored and Hadamard patterns are 
selected uniformly at random. 

• More complex samples have a power spectrum typically decay- 
ing like a power law. This a priori information suggests that 
we should balance low- and high-frequency measurements in or- 
der to accurately acquire the low-frequency part of the image, 
which accounts for a significant part of the total variance. The 
half-half strategy then projects the m/2 patterns with the lowest 
spatial frequencies to acquire a low-resolution image of the sam- 
ple; the high-resolution content of the image is randomly sampled 
by choosing m/2 measurements among the N — m/2 remain- 
ing high-frequency Hadamard patterns. Such an adaptive strategy 
guarantees an accurate determination of the low-frequency con- 
tent while allowing for the estimation of details at a finer scale. 

Computational reconstruction. In CS, it is essential to enforce the 
sparsity of the reconstructed signal in some representation W that is 
chosen a priori. The choice of W highly depends on the spatial 
structures of the signal to be reconstructed. One would typically use 
a Fourier representation for oscillatory features, wavelets for point- 
wise singularities, curvelets for contour-like or filamentary structures 
[18] and so on. One could also use a concatenation of all these rep- 
resentations. (If one intends on using the Fourier basis as a sparsity 
basis, one would need to scramble the columns of the Hadamard ba- 
sis since it would otherwise be coherent with sinusoids.) 

After recording the fluorescence intensity during a sequence of 
up to 65536 consecutive patterns (Figure C), one can imagine recov- 
ering the signal x from these data by solving the optimization prob- 
lem [3]. Because our measurements are noisy, it is actually better to 
relax the constraints into 

min ||W T x|| £l subject to ||y - *x|L < e; [5] 

we ask that the fit holds up to the noise level. In the following, W 
will be either an orthonormal basis (e.g. Dirac basis) or an over- 
complete signal representation (e.g. undecimated wavelet frame or 
curvelet frame). This will be clearly specified for each individual re- 
construction result. For computational reasons, we find it convenient 
to solve a relaxed version of this problem, namely, 

min ||W T x|| fl + ^||y-*x||, 2 [6] 

As is well known, there is a value a(e) such that the two programs 
coincide. For our experiments, we used the NESTA solver [30] and 
the regularization parameter a is chosen empirically depending on 
the noise level. When the signal is nearly sparse and the noise level 
low, it is known that this program finds a reconstruction with a low 
mean squared error (MSE). 

Sparse Fluorescence Images: Beads, Cells and Tissues 

Fluorescent beads. We first tested our CS microscope (with the 
20x objective) on a sample of fluorescent beads (diameter 2 /im, peak 
emission at 520 nm, Fluorospheres Invitrogen) deposited on a glass 
coverslip. At a low density of beads, the WF image is the super- 
position of a few fluorescence spots on a dark background, a signal 
similar to that of single molecule imaging data in biology [19]. As 
for the sparsity basis W, we obtained nearly equivalent results using 
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Fig. 2. Top left to bottom right: camera snapshot and reconstructed 256-by-256 bead images for values of the undersampling ratio equal to 8, 16, 32, 
64 and 128. (A) Plot of the PSNR (see text) for a nominal illumination level (blue curve) and for the same level reduced by a factor 10 (red curve) and a 
factor of 100 (green curve) (simulated data). The solid lines correspond to the PSNR in raster scan for the same surfacic illumination. (B) Same as (A) 
for the experimental data. 



the Dirac basis or a wavelet transform. Here we show images recon- 
structed with the wavelet transform and using a number of random 
256 x 256 Hadamard patterns decreasing from 16384 down to 512. 
(To be complete, we used a weighted l\ norm in eq. [ 6 ] where the 
weight of each coefficient is inversely proportional to scale.) In the 
following, the undersampling ratio is the ratio between the number TV 
of pixels and the number M of measurements. As shown in Figure 2, 
most of the bead positions are recovered with undersampling ratios 
up to 64, corresponding to M ~ 1.5% of N. At higher undersam- 
pling ratios, beads with low intensities are lost. 

To quantify the distorsion of the reconstructed image as a func- 
tion of the undersampling ratio, we calculated the Peak Signal-to- 
Noise Ratio, PNSR = 10 log(d 2 / M SE) where MSE = jV _1 ||x - 
x«f the squared distance between the reconstructed image from all 
the 256 x 256 possible measurements and that which only uses a frac- 
tion. Above, d is the dynamical range of the reconstruction obtained 
from a full sample. As shown in Figure 2A, the PSNR decreases with 
the undersampling ratio (blue curve) and seems to reach a plateau at 
ratios above 64 where most of the beads are lost. Since beads with 
low intensities are lost before brighter beads, we made a second set 
of measurements with an excitation light intensity divided by 100 to 
assess the effect of illumination on compression efficiency. The red 
curve in Figure 2A represents the PNSR of the reconstructed images 
as a function of the undersampling ratio. As expected, the PNSR is 
lower than that for the nominal illumination and reaches a plateau at 
an undersampling ratio of about 10, where almost all the beads are 
lost. This clearly shows that the distortion of the reconstructed image 
is strongly affected by the amount of detected fluorescent photons. 
Indeed at such low intensities, photon noise (also termed, shot noise) 
may be significant. 

To further explore the impact of photon noise on the compres- 
sion efficiency, we performed numerical simulations on an artificial 
image of fluorescent beads made of 50 Gaussian spots (FWHM 3 
pixels) randomly positioned in the field of view of size 256 x 256 
pixels. The simulated nominal illumination intensity Jo was set so 
that the resulting flux {i.e. the sum of the signals over all the pixels) 
was equal to fo = 6.4e3. Each measurement was then computed 
as one realization of a Poisson process with mean (t/>k, x). 



Reconstructions are processed with intensities Jo, io/10 and 
Jo/100 for a range of undersampling ratios between 2 and 64. As 
shown by the PSNR curves (Figure 2B), these simulations qualita- 
tively reproduce the loss of compression efficiency for low-light lev- 
els but fail to quantitatively estimate the PSNR of the reconstructed 
images. This suggests that photon noise is not the only source of im- 
age degradation in our imaging system. A possible additional cause 
is the discrepancies between the theoretical patterns and the effective 
illumination profiles in the sample plane. 



Lily anther slice. In order to investigate the potential of CFM for 
biological samples, we imaged slices of endogeneously fluorescent 
lily anther (Carolina Biological Supply). A conventional epifluores- 
cence image of a slice (Excitation 488nm/Emission 520 nm) recorded 
on a CCD camera can be seen in Figure 3 (upper left). The resolu- 
tion of this image has been sampled down to 128 x 128 pixels. We 
recorded the same image with the CFM setup by illuminating the 
sample with 16384 = 128 2 different Hadamard patterns (complete 
basis). For this experiment, we used a 20x lens with a 0.75 NA. Fur- 
ther, we used curvelets as sparsity basis W since they are known 
to sparsely encode contour-like structures together with the half-half 
strategy described earlier to account for both low-spatial frequencies 
shapes and higher frequencies details of the sample structure. (We 
again used a weighted l\ norm with weights inversely proportional 
to scale.) Reconstructed images with varying undersampling ratios 
from 1 to 8 are displayed in Figure 3 (top). Here, the method re- 
constructs images satisfyingly up to an undersampling ratio of about 
8. Compared to fluorescent beads, this lower figure can be primarily 
attributed to the lesser sparsity. 

Another important issue is that this sample is not fully two di- 
mensional. Due to the thickness of the slice (about 50 /im in this 
case, compared to the focal depth ~ 1/im), the contrast of Hadamard 
patterns diminishes away from the focal plane. This yields a non- 
modulated background signal and, as further discussed below, the 
photon noise associated to this signal affects the image reconstruction 
by "hiding" the useful information contained in the intensity fluctua- 
tions due to the variations in the illumination patterns (Figure IE). 
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Zyxin-mEOS2 COS7 cells. In many biological applications, it is 
essential to use high magnification and high NA optics and we thus 
aimed at testing CFM in these imaging conditions (oil-immersion 
objective 60x, NA 1,45). To overcome the limitations due to the 
short focal depth of a high NA objective, we used photoactivation 
techniques. COS7 cells were transfected with Zyxin-mEOS2 [21]. 
Zyxin is a protein mainly expressed in the cellular focal adhesions, 
at the surface on which the cells are plated. It was fused to mEOS2, 
a genetically-encoded photoconvertible fluorescent protein tag [20] 
widely used in super-resolution microscopy, that has green fluores- 
cence in its native state (Excitation 506nm / Emission 519nm) and 
can be converted to a red-emitting state (Excitation 573nm / Emis- 
sion 584nm) upon illumination with violet light. The COS7 cells 
were plated at density of 100000 cell/ml on 18mm coverslips on a 12 
well plate. The cells were transfected with Eos-Zyxin using chem- 
ical transfection (Fugene) 4-5 hours after plating and experiments 
were performed on live cells 18-30 hours after the transfection. By 
using an evanescent wave excitation with a laser at 405 nm, we could 
convert proteins situated at the vicinity (~ 100 nm) of the glass cov- 
erslip. Therefore, in our sample, the green emitting fluorophores 
are located within the 3D cellular volume while red-emitting pro- 
teins constitute a two-dimensional sample. The superimposed epi- 
fluorescence images in the green and red channels (256x256 pixels) 
are shown in Figure 3 (bottom, left). The same 2D ensemble of pho- 



toconverted proteins was subsequently imaged with the CFM setup 
with 32768 different 256x256 pixels Hadamard patterns (half of the 
full basis). For this set of data, the pixel size in the sample plane of 
each Hadamard pattern is 430 nm, about twice the diffraction limit. 
As for the bead images, we used a wavelet transform as sparsity basis 
W. 

The reconstructed images for undersampling ratios varying be- 
tween 2 and 15 are displayed in the second row of Figure 3. For 
direct comparison with the conventional epi-fluorescence WF image, 
a dual color image obtained by superposition of the red converted 
Zyxin-mEOS2 image and the green native Zyxin-mEOS2 image is 
shown in Figure 3. It is noteworthy that even if the fluorescence emis- 
sion of transfected COS7 cells is low compared to the lily anther slice 
(by about a factor of 10), the quality of the reconstructed CS images 
is good for undersampling ratios up to 8 and starts to be degraded at a 
ratio of about 15. The very low background of our two-dimensional 
sample clearly enables a better reconstruction. Since fluorescent pro- 
teins are very sensitive to photobleaching, one has to use low illumi- 
nation to minimize this effect during acquisition. Thus, photon noise 
effects, which manifested themselves only at reduced illumination 
intensities in the bead images, appear to be a limiting factor for CS 
imaging of less fluorescent and/or photo-damageable samples. The 
impact of photon noise on CFM is discussed in more details in a later 
section. 
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Fig. 3. Upper line: CS imaging of a slice of a lily anther. Left: Original image (128x128 pixels) by conventional epifluorescence microscopy. Left to 
right: the same sample imaged by CFM with undersampling ratios between 1 and 8. Lower line: CS imaging of COS7 cells expressing Zyxin-mEOS2. 
Left: superposition of the conventional epifluorescence images of the native (green) and converted form (red) of the markers. Left to right: CFM 
images of the converted form of the markers at undersampling ratios equal to 2, 4, 8 and 15. 



Hyperspectral Imaging in CFM 

Hyperspectral imaging is defined as the combined acquisition of spa- 
tial and spectral information. In biological imaging, a growing range 
of applications such as the study of protein localization and interac- 
tions require quantitative approaches that analyze several distinct flu- 
orescent molecules at the same time in the same sample [22]. These 
applications are in fact becoming ever more common with the avail- 



ability of an increasing panel of fluorescent dyes and proteins with 
emission ranging from the UV to the far red [23]. 

Multicolor data are usually acquired by selecting a few distinct 
spectral bands. However, in many cases, the incomplete separation 
of the different color channels due to the presence of autofluores- 
cence, along with cross-excitation and emission "bleed-through" of 
one color channel into the others render the interpretation of multi- 
band images difficult and/or ambiguous. To overcome these limita- 
tions, it would often be preferable to record the full spectral informa- 
tion at each pixel of the image. 
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In this context, two elements make the potential benefits of CS 
particularly appealing for hyperspectral measurements. First, a full 
data acquisition can take up a very long time since the number of 
voxels TV quickly gets very large. Second, the signal becomes com- 
parably sparser as the dimension increases. To demonstrate the possi- 
bility of CFM for hyperspectral fluorescent imaging, we modified the 
setup and replaced the point detector by a spectrometer coupled to an 
EMCCD camera (Evolve 512, Photometries USA). The entire spec- 
trum between 520nm and 640nm is recorded on 128x1 pixels. We 
spin-coated on a glass coverslip a mixture of 3 types of fluorescent 
beads (TransFluo Beads, Invitrogen, USA) with different emission 
spectra in our detection band (see Figure 4A for a gray WF image of 
the sample). A complete set of 256 x 256 Hadamard patterns was 
subsequently projected on the sample and, for each projected pattern, 
we recorded the fluorescence spectrum. 

The computational reconstruction of hyperspectral data can be 
performed in two different manners. The simplest one is a direct 
extension of the monochromatic case and consists in reconstructing 
each spectral band independently from the others. This approach, 
however, does not fully account for the particular structure of the hy- 
perspectral data. Rather, it is worthwhile to exploit sparsity in both 
the spatial and spectral domains. Hence, we propose a computational 
reconstruction by solving the same problem as in [ 6 ] with the follow- 
ing modification: x is now the full 2D-A data cube and <J> and W are 
waveforms <f>k[i, A] of both space and wavelength. In these exper- 
iments, W was obtained by tensorizing the Dirac basis in space — 
well adapted to point-wise structures like beads — with a wavelet ba- 
sis along the spectral dimension which is well suited for smooth vari- 
ations and occasional transients. A slice of the 2D-A sensing matrix 
at a fixed A is the same 2D partial Hadamard transform. 

We obtained full color images by pooling the data cube — see Fig- 
ure 4 — into 3 spectral bands: blue (500-530 nm), green (530-560 
nm) and red (560-630 nm). Such multicolor images are shown for 
varying values of the undersampling ratio in Figure 4. We observe 
that almost no degradation is seen for undersampling factors up to 
16. Furthermore, hyperspectral reconstructions provide the spectrum 
of each individual bead from the reconstructed 2D-A cube (the nor- 
malized spectra of 3 different beads are shown in 4F). The spectra 
are correctly reconstructed for undersampling factors up to 64. In- 
terestingly, when reducing the number of measurements, the distor- 
tion primarily affects the low intensity parts of the spectra, similar 



to the effect of increasing undersampling on the dimmer beads in 
monochromatic images. 
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Fig. 4. (A)-(E) Camera snapshot and reconstructed 256-by-256 bead im- 
ages for undersampling ratios equal to 1, 8, 16 and 32. (F) Normalized 
spectra (128 spectral lines) of 3 individual different beads circled in (B) 
for undersampling ratios equal to 1 (plain circles), 32 (squares) and 64 
(triangles). The grey area in the spectrum represents a rejection band of 
the dichroic mirror used in our setup. 

Discussion and Perspectives 

We have developed an imaging approach based on the concepts of 
CS theory which, on samples relevant for biological imaging, allows 
the reconstruction of fluorescence images with undersampling ratios 
up to 64. While this constitutes a significant gain over undersampling 
ratios achieved in prior CS-based imaging approaches, several factors 
(computational, instrumental or noise-related) still contribute to limit 
the current performances of CFM. Below, we discuss them as well as 
the prospects for future developments and applications. 

Point spread function and its modeling. In the current implemen- 
tation of CFM, we neglected the role of the point spread function 
(PSF) of the excitation pathway (corresponding to the lenses /i , f% 
and /3 and the objective) and used the idealized matrix with zeros and 
ones (depending whether a pixel is illuminated or not) for the inver- 
sion. In reality, the PSF acts as a low pass filter, meaning that those 
patterns illuminating the sample are spatially smoother than their the- 
oretical counterparts. In the framework of CS, it is well known that 
recovering a compressed signal from a sensing matrix that departs 
from the real one may, in general, dramatically degrade the recon- 
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struction quality. In our case, however, this approximation only has 
minor consequences. This holds for essentially two reasons. In our 
experiments, the individual pixel size of the Hadamard patterns was 
at least twice as large as the diffraction-limit of the microscope. In 
other words, the PSF only has a minor filtering impact on the pat- 
terns. Second, neglecting the effect of the PSF in the reconstruction 
does not dramatically reduce the quality of reconstruction. To see 
this, let f be the PSF of the lens so that the collected fluorescence is 
of the form y k = {<j)k * f , x) (we measure the dot product between 
the object of interest and the theoretical patterns convolved with the 
PSF). If the PSF is symmetric around the origin, we have 

y = $Fx 

in which F is the linear convolution with the PSF. Hence, neglecting 
the PSF recovers a signal x obeying y = <&x; in other words, one 
gets x = Fx which is x at the resolution of the microscope. Indeed, 
assuming that the lenses the excitation and emission PSFs are iden- 
tical, the CSM has the same resolution as its wide-field equivalent. 
Now if one wishes to account for the PSF, one would need to solve 
a joint decompression-deconvolution problem. If the deconvolution 
should provide a higher resolution image, it is well-known that it is 
usually at the cost of noise amplification. Here again, the sparsity 
prior used for decompression should of course help regularizing this 
deconvolution step. 

Noise and MSE. It is crucial to examine noise figures in our setup. 
There are two important facts affecting the quality of reconstruction: 
the noise distribution associated with CS-type data and the undersam- 
pling ratio. 

We assume below that photonic noise is the limiting source of 
noise, an assumption which sets CS-based optical systems apart from 
the common theoretical CS framework [24, 10]. To understand the 
important trade-offs, we compare the respective situations of RS and 
CS, two point detection imaging techniques. Below, we put Trs and 
7cs for the excitation intensity per unit area and per unit time during 
RS and CS acquisition. Likewise, Trs and Tcs are durations of exci- 
tation (for a single measurement). Finally, we set Ars = Irs x Trs 
and similarly for Acs- In practice, all these parameter values are ad- 
justed according to factors that one wants to optimize (acquisition 
speed, sensitivity, photobleaching and so on) and they must be eval- 
uated on a case by case basis. Hence, rather than an exhaustive com- 
parison of the relative merits of RS and CS acquisition, the discussion 
below aims at providing a general framework to understand the na- 
ture of the noise in CS measurements. 

In the case of RS, the ith pixel measurement is distributed as a 
Poisson random variable with mean and variance Ars x x[i]. Using 
the scaled observed data yields a per-pixel MSE equal to 

MSE(RS) = N' 1 ^E(z [i] - x[i]f = A^ 1 x x, 

i 

where x = N~ x l^\- F° r CS, suppose first that we collect all 
Hadamard measurements (no undersampling). Each measurement is 
an independent Poisson variable with mean A C s x (0k, x). With pat- 
terns of the form |(1 + hk) where hk is a Hadamard sequence, one 
can decompose the mean value of y k , as 

Acs x (0 k , x) = ^[jVx + (h k ,x)]. 

Hence, this is the sum of a DC offset and a Fourier-like compo- 
nent. The presence of the DC offset (which prevents the optical pat- 
terns from being negative) impacts the data SNR. Indeed, a possible 
source of concern is that for many high-frequency components, Nx 
may be much greater than the magnitude of (hk, x) — the DC com- 
ponent dominates the high-frequency coefficients. Therefore, when 
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measuring a high-frequency component, we need to deal with a large 
amount of noise coming from the average fluorescence of the sam- 
ple under study. This situation is arguably very different than in 
other applications — for instance, the acquisition of radio-frequency 
signals — where one can use sensing waveforms that take on negative 
values by switching the phase of the object we wish to acquire [7]. 
Inverting the Hadamard matrix gives a noisy image x[i] obeying 

2 

Ex[i] = x[i] and Var(x[i]) = - — x 

Acs 

(see the Appendix for details). In contrast to RS microscopy, we see 
that CFM yields a spatially invariant noise level in the pixel domain. 
In other words, by measuring the image projection on an incoherent 
basis, the noise gets spread equally over all the pixels. Hence, before 
applying any processing, the CS situation is more favorable for re- 
covering brighter areas but less so for dimmer regions. Summing up 
gives 

MSE(CS) = 2A CS ~ 1 x x. 

This analysis also shows that it is essential to minimize the sources of 
signal which could contribute to a constant background and increase 
x. This offset could be due to the nature of the sample itself but can 
also originate from stray light or out-of-focus fluorescence. These 
considerations explain, at least qualitatively, why the beads sample, 
where the background x is low compared to the bright spots and is 
purely bi-dimensional, is favorable for CFM. 

To consider the effect of statistical estimation procedures or data 
processing, consider the beads sample again in which sparsely dis- 
tributed beads are located in the field of view. Each bead is an iso- 
lated bright spot surrounded by wide non-fluorescent areas. Suppose 
then that we were to apply a thresholding estimator, setting to zero all 
intensities below a certain level, and keeping those above threshold. 
Then one would obtain a very low MSE in the CS setting since dark 
pixels would be correctly set to zero while bright pixels would have a 
variance that is orders of magnitude lower than that achievable in the 
RS case. In short, thresholding would effectively filter out the off- 
support noise and the situation would be extremely favorable to the 
CS approach. Quantitatively, if there are K bright pixels, the error 
after estimation for RS and CS would behave like 

MSE(RS) = Ars -1 x x, MSE(CS) = 2A CS ~ 1 x-xi. [7] 

Note the potentially enormous reduction in MSE by the factor K/N. 
Conversely, RS would be more effective for smooth and bright im- 
ages (sparse images in the frequency domain). 

The comparisons between RS and CS microscopy above are valid 
as long as the same illumination intensity per pixel is assumed and 
the number of measurements M is equal to the number of pixels. 
We now discuss the effect of undersampling. Now the CS recov- 
ery is both an inversion and a denoising algorithm and the recovery 
error depends on the compressibility of the signal (on how sparse 
it is). For instance, in the fully sampled case where sparsity is as- 
sumed in the spatial domain, the CS recovery would essentially invert 
the Hadamard matrix and then apply soft-thresholding to the output. 
Now suppose for simplicity that the signal is i^-sparse and that the 
number M of measurements is sufficient for perfect recovery from 
noiseless data. Then the squared recovery error from M noisy mea- 
surements as above would roughly scale like 

MSE(CS) = Co x Acs -1 x x x, [8] 

where Co is a small numerical constant (typically on the order of 
unity). Hence, the main difference with the MSE available from a 
full sample is a loss of a factor N/M — the undersampling ratio — 
in the MSE, compare [7]. In other words, halving the number of 
samples — everything else, namely, intensity and duration of excita- 
tion remaining the same — doubles the MSE. The same conclusion 
applies for approximately sparse signals for which the variance com- 
ponent of the MSE dominates the squared bias. Here again, the more 
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compressible the signal (the smaller K), the better the performance 
(e. g. the beads sample is favorable to CFM). We would like to also 
note that K/M is still much smaller than one so that even though we 
are sampling less, we may still end up with a much better MSE than 
in RS. 

To summarize, CFM is effective when: 1) the most informative 
parts of the sample are brighter than its mean value and 2) the sample 
is highly compressible. Notice that while these two requirements are 
sample dependent, the second gives some flexibility since one can 
select a representation in which a class of signals has an optimally 
sparse representation. 

Impact of the sample thickness. One important issue in CFM has 
to do with the patterned illumination of thick fluorescent samples. 
Indeed, with a wide-field linear excitation, the illumination propa- 
gates throughout the sample and causes the entire volume to fluo- 
resce. Since the optical transfer function of a circular aperture (such 
as a microscope lens) has a bandwidth that decreases with defocusing 
[1], the contrast of the pattern diminishes away from the focal plane. 
As a result, the fluorescence coming from out-of-focus planes is not 
modulated as a function of the patterns. In fact, this property serves 
as basis for optical sectioning in structured illumination microscopy 
(SIM) [25]. In the case of CFM, the off-focus signal contributes to an 
offset signal on the detector, which, as explained above, tends to sig- 
nificantly degrade the quality of the reconstruction. A few strategies 
can be considered to add sectioning capabilities to CS based imag- 
ing systems [26], [27] [15]. One recently demonstrated approach is 
based on the rejection of the off-focus signal, in a way similar to 
that of a programmable array microscope [15]. Another method is to 
avoid generating any off-focus signal at all. We demonstrated it for 
photoactivation using an evanescent-wave (Figure 3) and this can be 
extended to 2D activation within the sample volume with two-photon 
temporal focusing activation [28] or light sheet illumination [29]. In 
the long-term, an even better strategy is to illuminate with an inco- 
herent basis of 3D patterns and, subsequently, to directly reconstruct 
the sample in 3D. 

Conclusions and Prospect 

This paper presented the principles and implementation of compres- 
sive sensing in fluorescence microscopy together with its applications 
in biomedical imaging. Our approach, which is based on a patterned 
excitation of the sample combined with a point-detection of the emit- 
ted fluorescence, readily allows for substantial undersampling gains 
when compared to traditional raster-scanning approaches. It could 
also be useful in situations, such as a diffusing media, where direct 
imaging on a multi-pixel detector is not possible. Furthermore, we 
have set forth a distinctive prospect for hyperspectral acquisition, 
which has great potential for multicolor single molecule imaging. 
More generally, the acquisition of 3D, 4D (three spatial dimensions 
and one spectral or temporal dimension) or even higher dimensional 
signals puts unrealistic constraints on system resources. It is indeed 
hard to imagine that one would want to sample such huge data cubes 
at rates anywhere close to the Shannon rate. The key is that multi- 
dimensional signals become increasingly redundant in the sense that 
their information content grows at a much lower rate than the number 
of voxels. For example, movies are comparably far more compress- 
ible than still pictures. Likewise, hyperspectral movies are far more 
redundant than monochromatic movies, and so on. Expressed differ- 
ently, the ratio between the number of degrees of freedom and the 
number of voxels decreases very rapidly as the dimension increases. 
The extreme sparsity of higher dimensional signals cannot be ignored 



and we expect the advantages of CFM to become paramount in such 
applications. 

Appendix 

This short appendix justifies our SNR calculations, and we begin by 
introducing some notation. We denote by H the Hadamard matrix 
and by 1 the vector with all entries equal to one. Hence, acquiring 
all Hadamard patterns gives us independent Poisson variables with 
means 

d = Sx 



i(ll T + H)x. 



Here, we set Acs = 1 as the general case can be obtained via a simple 
rescaling. Hence, our estimate is of the form 



s-V, 



and it is easy to verify that 



2 /„ T N T > 
ye ieiy 



in which ei = (1,0,..., 0). (Observe that the first entry is a bit 
special here — a pixel at this location is always illuminated — and that 
we could always shift the Hadamard matrix as to select any special 
pixel.) Since Ey = d = Sx, we have Ex = x. Further, 

Cov(x) = S- 1 Cov(y)S- T , 

where Cov(x) is the covariance matrix of the random vector x. Since 
Cov(y) is the diagonal matrix with entries d = (di, di, . . . , (In), we 
have 

JV 

Var(xH) = ^|S- 1 [ l ,j]| 2 d J . 
j=i 

To proceed, one verifies that 

2 1, («,.?)/ (1, 1), 



and that J2f=i d i 
Plugging in gives 



Var(x[i]) 



\{N 2 x + Nx[l]) together with di = Nx. 



[N-2)x+±x[l], i = l, 
2x+^x[l], i£l 



(again this highlights the role played by the special pixel). Now sup- 
pose that the special pixel is chosen so that there is no probe at this 
location (which can always be arranged). Then we would know that 
x[l] = (and would not bother estimating the density at that loca- 
tion) and for i ^ 1, we would have 



Var(x[i\) = 2x 



as claimed. 
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